Field of the Invention
The present invention relates generally to interface devices and systems of the same between humans and computers, and more particularly to interface devices that provide force feedback to the user.
Description of the Related Art
Computer controlled simulations, games, and training are used extensively in many different industries. Applications can be executed on a vast array of available hardware platforms, from workstations to PCs to game consoles. These hardware types may collectively be referred to as computer systems within this description. Applications are designed to allow a user to interact in a virtual or simulated environment. A computer system displays a visual environment to the user on a display screen, and a user can interact with the simulated environment with a human-computer control input or interface device. Depending on a given hardware implementation, users may have many different types of control input or interface devices at their disposal to interact with a game or simulation. For example, force feedback interface devices can include remotes, joysticks, mice and keyboards, steering wheels, new novel controllers, and others. In some devices, “haptic” feedback is also provided to the user, more generally known as “force feedback.” These types of devices can provide physical sensations to the user of the input or interface device corresponding to the object being controlled in the game or simulation. However, these sensations have not been providing an accurate simulation experience.
In most cases, the physical sensations output by the interface device are achieved by the use of one or more active, passive or hybrid actuators. These actuators are controlled by the host computer system and are coupled to the interface device. Typically, the computer system receives sensor signals from the interface device and, in response, sends the appropriate force feedback signals to the actuators of the interface device in conjunction with simulation or game events. The actuators then provide forces on the interface device. The computer system can thus convey “physical sensations” to the user by means of an interface device in conjunction with visual and auditory representation of the object in a simulated environment. For example, in many applications, the feedback provided is in a rumble or vibration form.
The physical sensations—namely force feedback, conveyed by the interface device should mimic the simulated object's real life counterpart as closely as possible to provide an enjoyable, realistic and immersive experience. However, currently this feedback is not realistic. This type of vibration force feedback should indicate when a simulated entity has hit an object or some other experience, which may cause a bump or vibration. Unfortunately, these rumbles or vibrations are not comprehensive when providing some simulated experiences, like a driving experience simulation, as a driving experience is both based on the inputs provided by the user and the feedback of forces provided by the vehicle, such as movement of the steering wheel in response to interaction with the driving surface.
An example of a system that uses an interface device for use with a driving simulation is shown in FIG. 1. In FIG. 1, a force feedback system 10 includes a force feedback interface device 12 and a host computer 18. The illustrated system 10 can be used for a video game, training program or simulation, or other application. A user can manipulate an interface device 12 in one or more degrees of freedom of motion to control objects within an application on a host computer 18. Images are displayed on a display apparatus, such as screen 20, of the computer 18 in response to such manipulations. The computer 18 may be any device with a processing ability, such as a personal computer, workstation, video game console, or other computing or display device. Host computer 18 preferably implements a host application program with which a user is interacting via device 12 and other peripherals, if appropriate, and which can include force feedback functionality. The host computer 18 may run an application, such as a simulation, video game, virtual reality training program or application, or other application program that utilizes input from interface device 12 and outputs force feedback commands to the interface device 12.
Initially, prior art force feedback devices focused on providing maximum haptic fidelity. However, due to the costs associated with the hardware capable of providing the necessary performance, these devices were expensive and impractical for mainstream markets. Therefore, to make force-enabled devices more accessible to the mainstream mass market, more cost effective solutions have been utilized. These solutions use similar design approaches based on limitations related to the technology available to the mainstream consumer at the time, such as PCs and game consoles. This technology has limitations in the areas of communication bandwidth, processing power, and lack of a standardized language for communicating between the host application and interface device. Therefore, manufacturers devised solutions to overcome these limitations in the form of shared force effect processing or “dual-architecture” systems, and the creation of standardized application programming interfaces (“API”).
Dual-architecture systems were designed to accommodate the communication bandwidth limitations, as well as the limited processing power of host PCs, by moving the control aspects of the force feedback effects to the interface device. Providing the devices with a local microprocessor allows the storage, management, and computation of the force effects to be done locally. This on-device approach was pursued to reduce the latency effects from time loops associated with sending data to-and-from the device and host for computation.
The dual-architecture hardware system became an industry standard, especially with the introduction of the DirectInput component of the DirectX API from the Microsoft Corporation. DirectInput allowed for standardization of the architecture or system having low level high-frequency force-effects stored and implemented on a local microprocessor, while the high-level supervisory low-frequency force-commands are computed by the host application and transmitted to the interface device.
This standard is pervasive, in-part, due to the popularity of the Windows-based operating system for PCs. Therefore, most force feedback interface devices are now designed, based on DirectInput's capabilities. DirectInput, as a component of DirectX, allows communication in a standardized way with interface devices by utilizing lower levels of programming overhead. This standardization was further enhanced by the creation of a portfolio of force feedback “effects” universal to interface hardware, and easily implemented by application developers through the API. The effects defined in the API are a collection of predefined conditions and forces designed to be executed, based on the “movement” or “position” of the interface device. This creates two control modes of operation: rate control and position control. Rate control is an abstraction, which makes force feedback less intuitive because there is no direct physical mapping between the interface device motion and the commanded motion of the simulated computer entity or game entity. Position control refers to a mapping in which the displacement of the interface device directly dictates displacement of a simulated computer entity or game entity. Although, position control mapping is more intuitive than rate control, it is limited because the simulated entity cannot move unless the user's interface device object is in motion. This is yet another abstraction, and therefore another limitation to the levels of performance of force feedback technology in certain applications.
Currently, force feedback is not realistic and not consistent across devices. Various interface devices function differently and vehicle simulations are governed by the devices. Driving and vehicle simulations should feel uniform, based on the vehicle characteristic, regardless of the device used. Therefore calibration & adjustment of devices to a normalized setting to provide a consistent experience is desired. FIG. 2 shows an exemplary relationship 100 between the interface device 102 and in-game vehicle tires 104. Currently, displacement 106 of the device 102 causes displacement of the tires 108. However, this communication 110 is one directional. Events in the simulation or game do not cause displacement of the tires 108, and therefore the device 102 is not displaced because of in-game forces, instead only displaced by effects, which are passed to the device 102 as force feedback. For example, currently if force feedback is disabled in simulation, the vehicle wheels will not move in response to in-game or in-simulation events. Because force feedback is only provided to the wheel as an effect, not to provide realistic control of the vehicle, tire movement in response to in-simulation events or conditions are not used when force feedback is not on.
This is shown in FIG. 3, which shows an exemplary force feedback information loop 300. This loop is between the host computer or application 302 and the interface device 304. As shown, the interface device 304 sends position data 306 to the application 302. This position data 306 instructs the in-simulation vehicle or tires to move. The application then sends force feedback commands 308 back to the interface device 304. This is unrealistic as actual in-simulation interactions do not impact vehicle or interface device movement, therefore, the completion of this feedback loop, such that in-simulation events impact both vehicle control and the interface device is desirable.
The limitations of the prior art force feedback architecture become apparent in vehicle simulation, gaming, and training applications because the lack of realism, immersion, and overall “feel”. For example, in real life an interface device of a vehicle, a steering wheel, is part of a vehicle steering system, which is designed to impart changes in the direction of the vehicle. When the steering wheel is turned by the driver applying rotational force, it converts that force into a steering wheel displacement about its axis, which is governed by the steering ratio. The steering ratio is the ratio of the number of degrees turned at the steering wheel versus the number of degrees the front wheels are deflected about their axis. But, most importantly, steering ratio is also the mechanical advantage dictating the “force” required at the steering wheel necessary to deflect the front wheels about their axis. Therefore, force applied to the steering wheel, according to the real-time system dynamics, will dictate the amount of displacement of the front wheels. And, considering the link between the vehicle and steering wheel consists of a mechanically connected system operating in a closed-loop, the cause and effect of turning the steering wheel is equally valid in reverse. Therefore, “forces” acting on the front wheels, causing them to deflect about their steering axis, will translate through steering system dynamics into a corresponding change in rotational “position” and “force” upon the steering wheel. This bi-directional “position and force” translation in the steering system is governed by kinematic equations and vehicle-specific parameters and therefore not constant across all vehicles.
A problem evident when using prior art force feedback interface devices in vehicle simulation, gaming, and training applications is the current architecture's lack of capability to provide the needed bi-directional simultaneous control of “both” position and force. Considering the pervasiveness of the DirectInput model; the interface hardware, firmware, and host application software have all been shaped by its limitations. The shared-processing DirectInput model, even with advanced combinations of hardware and firmware, is still constrained by the original fundamental relationships inherent in the force feedback control loop. These relationships are illustrated when examining the prior art force feedback loop, as noted in FIG. 3 described above. As mentioned before, the predefined collection of DirectInput API effects are designed to be executed based on the “movement” or “position” of the interface device object—they are not designed to “control the position” of the interface device object. Therefore, as shown in FIG. 3, the current control loop architecture is incapable of bi-directional control, as needed for realistic vehicle simulation.
Additionally, in current simulations, each vehicle's kinematic relationship is the same due to the interface device dynamics (inertia, friction, damping, etc.). These should differ for each different vehicle. For example, currently the same force applied to an interface device, such as a ConstantForce signal of −5,000, will move the wheel the same distance for a formula car vs. a power-assisted SUV, which is unrealistic. Therefore, it is desirable to adjust the interpretation of device data for each vehicle. Additionally, vehicle dynamics should not vary for the same vehicle from device to device, as this also impacts vehicle performance. In simulations, the interface device dictates movement of a simulated vehicle's steering system. This is why there is a desire for “position feedback” in place of the traditional force feedback. The forces felt by a driver through the steering wheel or column are a by-product of the position. Also, with a ConstantForce signal, the application sends a force command and the rotation is based on wheel dynamics. Therefore, the same force command does not provide the same result with different devices.
Another problem evident in prior art haptic interface systems, when used with vehicle simulation, gaming, and training applications is the “offloading” of the simulation of steering rack kinematics and tire force dynamics to the interface device. As described above, the vehicle and steering wheel is a mechanically connected system operating in a closed-loop with a bi-directional “position and force” relationship dictated by vehicle-specific kinematic equations. However, currently, in simulations, that relationship is dictated by the unique mechanical dynamics of each specific interface device, which determines the device's response to standardized API commands. For example, the ConstantForce effect of the DirectInput library is an open loop vector force with a defined direction, magnitude, and duration, with a range between (−10,000 and 10,000). It is the effect most often used by host application developers to link physics-engine output of steering wheel forces to the interface device. However, a ConstantForce signal of (−5,000) for the exact same duration output by a host application, sent to two different manufacturer's interface devices, could rotate one device's steering wheel by −15° and the other by −45°, because of each interface device unique mechanical dynamics that output the same ConstantForce signal differently due to different motors, gearing, inertia, friction, power supply, etc.
This may be troublesome in the current force feedback loop architecture, as shown in FIG. 3, of first sending out a “force” demand, and then receiving back a steering wheel “position”. If the same application, with the same vehicle in simulation, generates a physics-engine derived force of −5 Nm, that force should translate through vehicle-specific kinematics to a defined amount of steering wheel rotation with corresponding force. However, based on the above example with the exact same signal, the steering wheel rotation can vary as much as −30° depending on the interface device used. This −30° variance in the interface device steering wheel position translates to a −2° (based on 15:1 steering ratio) variance in the deflection of the simulated vehicle's front wheels about the steering axis. To underscore this point, a simulated vehicle's steering system dynamics are therefore variable and dictated by the interface device's mechanical dynamics, yielding a system not based in reality. It is desirable to provide a more realistic and consistent system.
Another shortcoming of prior art haptic interface systems in vehicle simulation, gaming, and training applications is the subjective “mapping” of physics-engine forces into the scaled range acceptable for the standardized APIs, and one that corresponds to the mechanical dynamics of the interface device. The host application developer uses a physics-engine to calculate forces; however, these forces usually have no direct correlation to the standardized API force range and especially the force range capable to be generated on the interface device. Lack of mapping leads to force feedback clipping. For example, if a host application's physics-engine calculates a −5 Nm force rotating the steering wheel to the “left” about its axis of rotation, the developer's discretion is used as to what value of DirectInput's ConstantForce effect is appropriate for mapping, i.e., −500 or −5,000. This is compounded by the fact that each interface device, with its own unique mechanical dynamics, would output the mapped ConstantForce signal in a different way. Additionally, currently, host application software in vehicle simulations do not look at force saturation, as force layering and force saturation are an issue, which is not being addressed.
Even though prior art force feedback interfaces and force feedback control systems have attempted to provide quality haptic feedback in vehicle simulation, gaming and training applications, the absence of vehicle-specific kinematic relationships, the lack of integration between interface hardware and host application software, and the limitations of the standardized force feedback application program interfaces (“API”), has led to a continued lack of realism and immersion for users in certain applications. Therefore, improvements and new approaches on these techniques are desirable. A need exists for a force feedback interface system that can allow the host application to function according to its intended design without regard to the dynamics of the interface device. A need exists for a system that can accommodate existing and future control input devices that can be readily connected to, or embedded within said devices. Finally, a need exists for a complete system that is realistic, accurate, reprogrammable, expandable, and most importantly, easily customizable by the user.